test(ingestion): cover the Dagster assets - #861
Merged
Conversation
ingest.py was at 16%. Every part of san_acacia_observations was covered on its own -- matching, resolving, watermarks, the adapter, the loader -- but the orchestration between them was not, which is where the decisions live: which wells get skipped, what the metadata reports, and whether one unresolvable well costs the others. Now 87%, and automated_ingestion overall 83% to 88%. Fakes stand in at the process boundaries only -- the vendor client, the database session, the dlt pipeline. The reconciler, resolver and adapter run for real, so a change in their behaviour surfaces here rather than being absorbed by a mock. Covered: a well with no match is skipped rather than invented; two wells sharing a name are skipped rather than picked between; a well with no open transducer, and one whose transducer was removed, are skipped; a refused reading is counted rather than vanishing; one bad well does not cost the others; both raw assets write parquet; a point the vendor refuses is counted and reported. What remains uncovered is _client and the two SQLAlchemy query helpers. They are thin wrappers around a live database and a real HTTP session, and testing them would mean asserting that mocks were called. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Coverage✅ 78.59% total — gate is 75%. No measured coverage for the Python files changed here. |
Contributor
|
Your pull request is automatically being deployed to Dagster Cloud.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ingest.pywas at 16%. Now 87%;automated_ingestionoverall 83% → 88%.Every part of
san_acacia_observationswas already covered on its own —matching, resolving, watermarks, the adapter, the loader. What was not covered
was the orchestration between them, which is where the decisions live: which
wells get skipped, what the metadata reports, and whether one unresolvable well
costs the others.
Fakes only at the process boundaries
The vendor client, the database session, and the dlt pipeline are faked. The
reconciler, resolver and adapter run for real, so a change in their
behaviour surfaces here rather than being absorbed by a mock.
What is now pinned
What remains uncovered, deliberately
_clientand the two SQLAlchemy query helpers (_well_candidates,_deployments). They are thin wrappers around a live database and a real HTTPsession — testing them would mean asserting that mocks were called, which proves
nothing about whether the queries are right.
The honest measure of what is still unproven is that the pipeline has never run
end to end against the database. These tests do not change that; they make the
skip-and-report logic verifiable without it.
124 tests pass.
🤖 Generated with Claude Code